Virtual-world simulator

ABSTRACT

In one implementation, a virtual-world simulator includes a computing platform having a hardware processor and a memory storing a software code, a tracking system communicatively coupled to the computing platform, and a projection device communicatively coupled to the computing platform. The hardware processor is configured to execute the software code to obtain a map of a geometry of a real-world venue including the virtual-world simulator, to identify one or more virtual effects for display in the real-world venue, and to use the tracking system to track a moving perspective of one of a user in the real-world venue or a camera in the real-world venue. The hardware processor is further configured to execute the software code to control the projection device to simulate a virtual-world by conforming the identified one or more virtual effects to the geometry of the real-world venue from a present vantage point of the tracked moving perspective.

BACKGROUND

In augmented reality (AR), the appearance of a real-world environmentcan be digitally modified to provide a user with the sensation ofengaging with a virtual-world. AR is increasingly used to produceentertainment experiences that are more immersive and engaging.Moreover, AR can be used to modify images of the real-world throughaugmentation in ways that have practical applications. Nevertheless, auser wishing to enjoy or otherwise utilize a virtual environmentgenerated using AR must typically view real-world objects through theviewport of an AR enabled personal device, such as AR glasses orgoggles, an AR headset, or a suitably configured smartphone or tabletcomputer, in order to see those real-world objects overlaid by virtualprojections. Moreover, conventional approaches to generating AR imageryproduce two-dimensional (2D) digital augmentations to three-dimensional(3D) real-world objects.

Despite their inability to provide a true 3D virtual experience, ARglasses, goggles, and headsets can be costly and inconvenient to wear.In addition, the increased concern over the spread of communicabledisease will likely mandate burdensome sanitation procedures in usageenvironments in which wearable AR viewing equipment is shared bymultiple users. Furthermore, requiring the use of an AR enabled personaldevice to enjoy a virtual environment effectively precludes multipleusers from sharing the same experience. Consequently, there is a need inthe art for a solution enabling one or more users to experience animmersive simulation of a 3D virtual-world that is accurately renderedaccording to the vantage point of each user.

SUMMARY

There are provided virtual-world simulators and methods for use by suchsimulators, substantially as shown in and/or described in connectionwith at least one of the figures, and as set forth more completely inthe claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A shows a diagram of an exemplary virtual-world simulator,according to one implementation;

FIG. 1B shows a diagram of an exemplary virtual-world simulator,according to another implementation;

FIG. 1C shows a diagram of an exemplary virtual-world simulator,according to another implementation;

FIG. 1D shows a diagram of an exemplary virtual-world simulator,according to yet another implementation;

FIG. 2 shows a diagram including a more detailed exemplaryrepresentation of a virtual-world simulator, according to oneimplementation; and

FIG. 3 is a flowchart presenting an exemplary method for use by avirtual-world simulator, according to one implementation.

DETAILED DESCRIPTION

The following description contains specific information pertaining toimplementations in the present disclosure. One skilled in the art willrecognize that the present disclosure may be implemented in a mannerdifferent from that specifically discussed herein. The drawings in thepresent application and their accompanying detailed description aredirected to merely exemplary implementations. Unless noted otherwise,like or corresponding elements among the figures may be indicated bylike or corresponding reference numerals. Moreover, the drawings andillustrations in the present application are generally not to scale, andare not intended to correspond to actual relative dimensions.

The present application discloses virtual-world simulators and methodsfor use by such simulators that overcome the drawbacks and deficienciesin the conventional art. According to the present novel and inventiveconcepts, one or more users can advantageously experience avirtual-world that is simulated by conforming virtual effects to thethree-dimensional (3D) geometry of a real-world venue from the vantagepoint of each user. It is noted that, as used herein, the feature“virtual effect” refers to one or more virtual images used to overlay animage of a real-world object. Moreover. “virtual effect” refers to oneor more virtual images in the form of environmental features, such aslighting, color, or structural/architectural features of a venue, or tosimulations of persons, avatars, characters, caricatures of a person,animals, plants, and living things of various species or varieties, aswell as inanimate objects.

In some implementations, the virtual-world simulators and methodsdisclosed by the present application may be substantially or fullyautomated. It is noted that, as used in the present application, theterms “automation.” “automated.” and “automating” refer to systems andprocesses that do not require the participation of a human systemoperator. Although in some implementations a human user may makeadjustments to the automated systems described herein that humaninvolvement is optional. Thus, in some implementations, the methodsdescribed in the present application may be performed under the controlof hardware processing components of the disclosed virtual-worldsimulators.

FIG. 1A shows a diagram of exemplary virtual-world simulator 100A,according to one implementation. As shown in FIG. 1A, virtual-worldsimulator 100A includes computing platform 110 having hardware processor112, and memory 114 implemented as a non-transitory storage devicecontaining virtual effects conforming software code 118. In addition,virtual-world simulator 100A includes tracking system 102 and projectiondevice 104 each communicatively coupled to computing platform 110. Asfurther shown in FIG. 1A, in some implementations, virtual-worldsimulator 100A may also include mapping device 106 communicativelycoupled to computing platform 110.

It is noted that, as defined for the purposes of the presentapplication, the expression “communicatively coupled” may meanphysically integrated with, or physically discrete from but incommunication with. Thus, one or more of tracking system 102, projectiondevice 104, and mapping device 106 may be integrated with computingplatform 110, or may be adjacent to or remote from computing platform110 while being in wired or wireless communication with computingplatform 110.

As further shown in FIG. 1A, virtual-world simulator 100A is implementedwithin real-world venue 120 including one or more objects or structures,represented by floor 122, wall 124, object 126 a, and object 126 bhaving curved surface 128. Also shown in FIG. 1A is virtual-worldsimulation 130 produced using projection device 104 of virtual-worldsimulator 100A, and user 132 engaging with virtual-world simulation 130.

In some implementations, real-world venue 120 may take the form of apersonal residence, a theme park attraction, a game environment, or afilm or broadcast studio, to name a few examples. It is noted thatalthough FIG. 1A explicitly shows real-world venue 120 to include onlyfloor 122, wall 124, and objects 126 a and 126 b, that simplifiedrepresentation is provided merely for conceptual clarity. Moregenerally, real-world venue 120 may include multiple structures, such asadditional walls, a ceiling, and one or more additional objects, such asarticles of furniture and/or art or other decorative objects, forexample.

It is noted that, although the present application refers to virtualeffects conforming software code 118 as being stored in memory 114 forconceptual clarity, more generally, memory 114 may take the form of anycomputer-readable non-transitory storage medium. The expression“computer-readable non-transitory storage medium,” as used in thepresent application, refers to any medium, excluding a carrier wave orother transitory signal that provides instructions to hardware processor112 of computing platform 110. Thus, a computer-readable non-transitorymedium may correspond to various types of media, such as volatile mediaand non-volatile media, for example. Volatile media may include dynamicmemory, such as dynamic random access memory (dynamic RAM), whilenon-volatile memory may include optical, magnetic, or electrostaticstorage devices. Common forms of computer-readable non-transitory mediainclude, for example, optical discs, RAM, programmable read-only memory(PROM), erasable PROM (EPROM), and FLASH memory.

Tracking system 102 may include a camera, camera array, or one or moreother types of optical sensors for determining the moving perspective ofuser 132 as user 132 moves around in real-world venue 120. As anotheralternative, or in addition, tracking system 102 may include multiplecomponents distributed within real-world venue 120 and configured toperform radio-signal triangulation to determine the moving perspectiveof user 132. As yet another alternative, computing platform 110 mayutilize optional mapping device 106 configured to perform simultaneouslocalization and mapping (SLAM) to determine the moving perspective ofuser 132 in real-world venue 120.

It is noted that, as defined for the purposes of the presentapplication, the expression “moving perspective,” as it applies to user132, refers to the eye-point perspective or viewing perspective of user132. Thus, for example, in some implementations, tracking system 102 maybe configured to perform head tracking, eye tracking, or skeletontracking of user 132 in real-world venue 120. Alternatively, or inaddition, in some implementations tracking system 102 may be configuredto perform hand tracking of user 132 in order to enable detection ofgestures, such as a pointing motion, performed by user 132.

In some implementations, projection device 104 may include one or moreprojectors configured not only to project virtual-world simulation 130onto real-world venue 120, but to conform the virtual effects includedin virtual-world simulation 130 to the 3D geometry of real-world venue120 from the vantage point of user 132 as user 132 moves withinreal-world venue 120. By way of example, hardware processor 112 ofcomputing platform 110 may execute virtual effects conforming softwarecode 118 to control projection device 104 to warp or otherwise distort avirtual effect projected onto object 126 b of real-world venue 120 toconform the virtual effect to curved surface 128 of object 126 b fromthe present vantage point of user 132, i.e., the location of the movingperspective of user 132 at the time of projection of the virtual effect,and to vary that conformal projection in real-time as the vantage pointof user 132 changes. As a result, virtual-world simulator 100A canadvantageously generate virtual-world simulation 130 providing user 132with a realistic and highly immersive 3D virtual experience withoutrequiring user 132 to wear an augmented reality (AR) viewing device,such as an AR headset or glasses, or to carry an AR enabled personalcommunication device, such as a smartphone or tablet computer.

FIG. 1B shows a diagram of exemplary virtual-world simulator 100B,according to another implementation. It is noted that virtual-worldsimulator 100B, in FIG. 1B, corresponds in general to virtual-worldsimulator 100A, in FIG. 1A, and may share any of the characteristicsattributed to that corresponding simulator by the present disclosure. Itis further noted that any feature in FIG. 1B identified by a referencenumber identical to a reference number appearing in FIG. 1A correspondsto that previously described feature and may share any of thecharacteristics attributed to it above.

According to the exemplary implementation shown in FIG. 1B, and incontrast to the implementation shown in FIG. 1A, the moving perspectivebeing tracked in FIG. 1B is the moving perspective of camera 134 ascamera 134 is moved within real-world environment 120 to obtain imagesof user 132 in virtual-world simulation 130. It is noted that camera 134is not a component of virtual world simulator 100B, i.e., camera 134 isnot part of tracking system 102. It is further noted that, as definedfor the purposes of the present application, the expression “movingperspective.” as it applies to camera 134, refers to the image captureperspective or viewing perspective of camera 134. Thus, for example, insome implementations, tracking system 102 may be configured to track thelocation and orientation of camera 134 in real-world venue 120. Camera134 may include one or more red-green-blue (RGB) still image camerasand/or video cameras. Moreover, in some implementations, camera 134 maycorrespond to an array of RGB still image and/or video camerasconfigured to generate a panoramic image of a venue, such as real-worldvenue 120.

In the exemplary implementation shown in FIG. 1B, for example, hardwareprocessor 112 of computing platform 110 may execute virtual effectsconforming software code 118 to control projection device 104 to warp orotherwise distort virtual effects projected onto real-world venue 120 toconform the virtual effects to the 3D geometry of objects withinreal-world venue 120 from the present vantage point of camera 134, i.e.,the location of the moving perspective of camera 134 at the time ofprojection of the virtual effect, and to vary that conformal projectionin real-time as the vantage point of camera 134 changes. As a result,virtual-world simulator 100B can advantageously generate virtual-worldsimulation 130 enabling transformation of real-world venue 120 in theform of a personal residence or photography studio into a movie setproviding a backdrop for a performance by user 132, without requiringuser 132 to be present at a movie studio backlot, or even to leavehis/her own home.

FIG. 1C shows a diagram of exemplary virtual-world simulator 100C,according to another implementation. It is noted that virtual-worldsimulator 100C, in FIG. 1C, corresponds in general to virtual-worldsimulator 100A and 100B, in FIGS. 1A and 1B, respectively, and may shareany of the characteristics attributed to that corresponding simulator bythe present disclosure. It is further noted that any feature in FIG. 1Cidentified by a reference number identical to a reference numberappearing in FIG. 1A or 1B corresponds to that previously describedfeature and may share any of the characteristics attributed to it above.In addition to the features shown and described by reference to FIGS. 1Aand 1B, FIG. 1C includes second user 136 and may optionally include userdevice 140 having display 148.

According to the exemplary implementation shown in FIG. 1C, and incontrast to the implementations shown in FIGS. 1A and 1B, virtual-worldsimulator 100C is configured to use tracking system 102 to track themoving perspective of second user 136, while also tracking the movingperspective of user 132, and to generate a virtual-world simulationconforming the virtual effects included in virtual-world simulation 130to the geometry of real-world venue 120 from the present vantage pointof the second moving perspective of second user 136 while concurrentlyproviding real-world simulation 130 from the present vantage point ofthe moving perspective of user 132. In some implementations, projectiondevice 104 may be a high-frame-rate projector, or a polarized projectiondevice, for example, configured to project multiple perspectives ofvirtual-world simulation 130 onto real-world venue 120 so as to causethose multiple perspectives to appear to be produced concurrently inreal-time to user 132 and second user 136.

In other implementations, virtual-world simulator 100C may communicatewith user device 140 operated by second user 136 to provide second user136 with virtual-world simulation 130 conformed to the geometry ofreal-world venue 120 from the vantage point of second user 136 viadisplay 148 of user device 140. User device 140 may take the form of asmartphone or tablet computer, as depicted in FIG. 1C, or may beimplemented as an AR viewer such as an AR headset, goggles, or glasses.Alternatively, in some implementations, user device 140 may be anothertype of smart wearable device including display 148, such as asmartwatch.

In one implementation, hardware processor 112 of computing platform 110may execute virtual effects conforming software code 118 to detect userdevice 140 utilized by second user 136 of real-world venue 120, to trackthe second moving perspective of second user 136 in real-world venue 120using tracking system 102, and to generate a virtual-world simulationfor second user 136 by conforming the virtual effects included invirtual-world simulation 130 to the geometry of real-world venue 120from the present vantage point of the moving perspective of second user136. In that implementation, hardware processor 112 of computingplatform 110 may further execute virtual effects conforming softwarecode to transmit the generated simulation to user device 140 for displayto second user 136 via display 148, while concurrently projectingvirtual-world simulation 130 from the present vantage point of themoving perspective of user 132. It is noted that display 148 may takethe form of a liquid crystal display (LCD), a light-emitting diode (LED)display, an organic light-emitting diode (OLED) display, or using anyother suitable display technology that performs a physicaltransformation of signals to light.

FIG. 1D shows a diagram of exemplary virtual-world simulator 100D,according to yet another implementation. It is noted that virtual-worldsimulator 100D, in FIG. 1D, corresponds in general to virtual-worldsimulator 100A, 100B, and 100C, in FIGS. 1A, 1B, and 1C, respectively,and may share any of the characteristics attributed to thatcorresponding simulator by the present disclosure. It is further notedthat any feature in FIG. 1D identified by a reference number identicalto a reference number appearing in FIG. 1A. 1B, or 1C corresponds tothat previously described feature and may share any of thecharacteristics attributed to it above.

According to the exemplary implementation shown in FIG. 1D, user device140 may be a handheld device configured to project virtual-worldsimulation 130 onto real-world venue 120. It is noted that in theexemplary implementation shown in FIG. 1D, user device 140 may beincluded as a remote component of virtual-world simulator 100D,communicatively coupled to computing platform 110. In thoseimplementations, user device 140 may include projection device 104 shownin FIGS. 1A, 1B, and 1C. That is to say, in some implementations theprojection device of virtual-world simulator 100D may be handheld userdevice 140 configured for use by user 132 of real-world venue 120.Alternatively, in some implementations, virtual-world simulator 100D maybe integrated into user device 140, thereby imbuing user device 140 withsubstantially all of the features and functionality of any ofvirtual-world simulators 100A, 100B. 100C, and 100D (hereinafter“virtual-world simulators 100A-100D”) disclosed in the presentapplication.

FIG. 2 shows a more detailed representation of exemplary user device 240in combination with computing platform 210 of virtual-world simulator200, according to one implementation. As shown in FIG. 2, user device240 is communicatively coupled to computing platform 210. Computingplatform 210 includes hardware processor 212, and memory 214 implementedas a non-transitory storage device. In addition, in someimplementations, computing platform 210 may include one or more ofprojection device 204 a, mapping device 206 and transceiver 216. Asfurther shown in FIG. 2, memory 214 contains virtual effects conformingsoftware code 218 a, virtual-world database 250 storing virtual effects256, and may further contain map database 266 having stored therein map258 of the geometry of real-world venue 120 shown in FIGS. 1A, 1B, 1C,and 1D (hereinafter “FIGS. 1A-1D”).

User device 240 includes hardware processor 242, and memory 244implemented as a non-transitory storage device storing virtual effectsconforming application 218 b. As also shown in FIG. 2, user device 240may include one or more of transceiver 246, one or more cameras 260(hereinafter “camera(s) 260”), user projection device 204 b, one or moreposition/location sensors 262 (hereinafter “P/L sensor(s) 262”). Alsoshown in FIG. 2 is virtual-world simulation 230, which, as discussedbelow, may be generated on computing platform 210 and projected usingprojection device 204 a, may be generated on computing platform 210 andbe transmitted to user device 240 for display on display 248 or forprojection by user projection device 204 b, or may be generated on userdevice 240 for display on display 248 or for projection by userprojection device 204 b.

Virtual-world simulator 200 including computing platform 210 havinghardware processor 212, projection device 204 a, optional mapping device206, and memory 214 including virtual effects conforming software code218 a, virtual-world database 250, and map database 266 corresponds ingeneral to any of virtual-world simulators 100A-100D including computingplatform 110 having hardware processor 112, projection device 104,optional mapping device 106, and memory 114 including virtual effectsconforming software code 118, variously shown in FIGS. 1A-1D. Thus,computing platform 210, hardware processor 212, projection device 204 a,optional mapping device 206, memory 214, and virtual effects conformingsoftware code 218 a may share any of the characteristics attributed torespective computing platform 110, hardware processor 112, projectiondevice 104, optional mapping device 106, memory 114, and virtual effectsconforming software code 118 by the present disclosure, and vice versa.Moreover, it is noted that although not shown in FIG. 2, likevirtual-world simulators 100A-100D, computing platform 210 ofvirtual-world simulator 200 is communicatively coupled to a trackingsystem corresponding to tracking system 102. It is further noted thatvirtual-world simulation 230, in FIG. 2, corresponds in general tovirtual-world simulation 130 in FIGS. 1A-1D, and those correspondingfeatures may share any of the characteristics attributed to eithercorresponding feature herein.

User device 240 including display 248 corresponds in general to userdevice 140 including display 148, in FIGS. 1C and 1D, and thosecorresponding features may share any of the characteristics attributedto either corresponding feature by the present disclosure. Thus, likeuser device 240, user device 140 may include features corresponding tohardware processor 242, transceiver 246, camera(s) 260, P/L sensor(s)262, user projection device 204 b, and memory 244 storing virtualeffects conforming application 218 b. In addition, like user device 140,user device 240 may take a variety of forms. For example, as describedabove by reference to FIGS. 1C and 1D, user device 140/240 may be ahandheld device, such as a handheld projection device, or a smartphoneor tablet computer. Alternatively, and as also described above, in someimplementations, user device 140/240 may be a wearable device, such as asmartwatch, or an AR headset, goggles, or glasses. Moreover, likedisplay 148, display 248 may be an LCD, an LED display, an OLED display,or a display using any other suitable display technology that performs aphysical transformation of signals to light.

Transceiver 216 and/or transceiver 246 may be implemented as wirelesscommunication hardware and software enabling user device 140/240 toexchange data with computing platform 110/210. For example, transceiver216 and/or transceiver 246 may be implemented as fourth generation ofbroadband cellular technology (4G) wireless transceivers, or as 5Gwireless transceivers configured to satisfy the IMT-2020 requirementsestablished by the International Telecommunication Union (ITU).Alternatively, or in addition, transceiver 216 and/or transceiver 246may be configured to communicate via one or more of WiFi, Bluetooth,ZigBee. and 60 GHz wireless communications methods.

Camera(s) 260 may include one or more RGB still image cameras and/orvideo cameras. Moreover, in some implementations, camera(s) 260 maycorrespond to an array of RGB still image and/or video camerasconfigured to generate a panoramic image of a venue, such as real-worldvenue 120. P/L sensor(s) 262 may include one or more accelerometers,and/or gyroscopes, and/or a GPS receiver, and/or a magnetometer, forexample. In some implementations, P/L sensor(s) 262 may be implementedas an inertial measurement unit (IMU), as known in the art.

With respect to virtual effects conforming application 218 b, it isnoted that in some implementations, virtual effects conformingapplication 218 b may be a thin client application of virtual effectsconforming software code 118/218 a. In those implementations, virtualeffects conforming application 218 b may enable user device 140/240 toobtain one or more of virtual effects 256, map 258 of the geometry ofreal-world venue 120, and/or virtual-world simulation 130/230. Moreover,in some implementations, virtual effects conforming application 218 b,executed by hardware processor 242 of user device 140/240, may track thelocation of user 132 in real-world venue 120 using P/L sensor(s) 262,and may communicate that tracked location to computing platform 110/210.

However, in other implementations, virtual effects conformingapplication 218 b may include substantially all of the features andfunctionality attributed to virtual effects conforming software code118/218 a by the present application. Moreover, in some of thoseimplementations, user device 140/240 may serve as the computingplatform, and/or projection device, and/or tracking system, and/ormapping device of virtual-world simulator 100/200. In other words, insome implementations, virtual-world simulator 100/200 may beincorporated into user device 14/240.

The functionality of virtual effects conforming software code 118/218 aand virtual effects conforming application 218 b will be furtherdescribed by reference to FIG. 3 in combination with FIGS. 1A-1D and 2.FIG. 3 shows flowchart 370 presenting an exemplary method for use by avirtual-world simulator. With respect to the method outlined in FIG. 3,it is noted that certain details and features have been left out offlowchart 370 in order not to obscure the discussion of the inventivefeatures in the present application.

Referring to FIG. 3 in combination with FIGS. 1A-1D, and 2, flowchart370 begins with obtaining map 258 of the geometry of real-world venue120 (action 372). In some implementations, map 258 may be a previouslyproduced 3D map of the geometry of real-world venue 120 and may bestored in map database 266, or may have been previously produced and beobtainable from a third party or other remote provider. In some usecases in which map 258 already exists, hardware processor 112/212 ofcomputing platform 110/210 may execute virtual effects conformingsoftware code 118/218 a to obtain map 258 from map database 266 storedin memory 114/214 of computing platform 110/210. Alternatively, in someuse cases in which map 258 already exists, hardware processor 142/242 ofuser device 140/240 may execute virtual effects conforming softwareapplication 218 b to obtain map 258 from computing platform 110/210.

As noted above, in some implementations, virtual-world simulators100A-100D/200 may include optional mapping device 106/206. Mappingdevice 106/206 may include a camera, such as a three hundred and sixtydegree (360°) camera, a camera array, or one or more other types ofoptical sensors for mapping real-world venue 120. Alternatively, or inaddition, mapping device 106/206 may include a light detection andranging (LIDAR) device for mapping real-world venue 120. Thus, in someimplementations, obtaining map 258 of the geometry of real-world venue120 may be performed by virtual effects conforming software code 118/218a of computing platform 110/210, executed by hardware processor 112/212,and using mapping device 106/206 to generate map 258 of the geometry ofreal-world venue 120. Moreover, in some implementations, map 258 may begenerated as a 3D map using mapping device 106/206.

It is noted that, in some implementations, virtual-world simulators100A-100D/200 may be configured to use mapping device 106/206 togenerate map 258 of the geometry of real-world venue 120 substantiallyin real-time with respect to the moving perspective of user 132, camera134, or second user 136. In those implementations, map 258 canadvantageously be updated to include dynamic changes to the geometry ofreal-world venue 120, such as the movement of furniture or other objectswithin real-world venue 120 during the presence of user 132, camera 134,or second user 136 in real-world venue 120.

Flowchart 370 continues with identifying one or more virtual effects 256for display in real-world venue 120 (action 374). Virtual effects 356may include a virtual background environment, as well as one or morevirtual assets in the form of simulated human or animated characters, orsimulated furniture, artwork, or other objects or props, for example. Insome implementations, action 374 may be performed by virtual effectsconforming software code 118/218 a of computing platform 110/210,executed by hardware processor 112/212, and using virtual-world database250 stored in memory 114/214 of computing platform 110/210.Alternatively, in other implementations, hardware processor 142/242 ofuser device 140/240 may execute virtual effects conforming softwareapplication 218 b to identify one or more virtual effects 256 stored invirtual-world database 250 and to obtain virtual effects 256 fromcomputing platform 110/210.

Flowchart 370 continues with tracking the moving perspective of user 132of real-world venue 120 or the moving perspective of camera 134 inreal-world venue 120 (action 376). As discussed above, virtual-worldsimulator 100A-100D/200 may include tracking system 102 communicativelycoupled to computing platform 110/210. Tracking system 102 may beconfigured to track the moving perspective of user 132 or camera 134 inreal-world venue 120. For example, and as also discussed above, trackingsystem 102 may include a camera, camera array, or one or more othertypes of optical sensors for tracking the perspective of user 132 orcamera 134 as user 132 or camera 134 moves within real-world venue 120.

As another alternative, or in addition to the use of one or morecameras, tracking system 102 may include multiple components distributedwithin real-world venue 120 and configured to perform radio-signaltriangulation to track the moving perspective of user 132 or camera 134in real-world venue 120. As yet another alternative, computing platform110/210 may utilize optional mapping device 106/206 configured toutilize a SLAM technique to track the moving perspective of user 132 orcamera 134 in real-world venue 120. Thus tracking of the movingperspective of user 132 or camera 134 in real-world venue 120 usingtracking system 102 and/or mapping device 106/206 may be performed byvirtual effects conforming software code 118/218 a of computing platform110/210, executed by hardware processor 104.

Alternatively, or in addition, in some implementations user device140/240 may include P/L sensor(s) 262, and may be configured to monitorits own position and orientation in real-world venue 120, and to reportthat position and orientation to computing platform 110/210. In thoseimplementations, user device 140/240 may utilize transceiver 246 toself-report its position and orientation in real-world venue 120. Insome implementations, tracking of the moving perspective of user 132 orcamera 134 in real-world venue 120 in response to self-reporting by userdevice 140/240 may be performed by virtual effects conforming softwarecode 118/218 a of computing platform 110/210, executed by hardwareprocessor 112/212. However, in other implementations, tracking of themoving perspective of user 132 or camera 134 in real-world venue 120 maybe performed by virtual effects conforming application 218 b, executedby hardware processor 242 of user device 140/240, and based on data fromP/L sensor(s) 262.

Flowchart 370 can conclude with controlling projection device 104/204 aor user projection device 204 b to project virtual-world simulation130/230 by conforming the one or more virtual effects identified inaction 374 to the geometry of real-world venue 120 from the presentvantage point of the moving perspective of user 132 or camera 134tracked in action 376 (action 378). Referring to FIGS. 1A and 2, forexample, hardware processor 112/212 of computing platform 110/210 mayexecute virtual effects conforming software code 118/218 a to controlprojection device 104/204 a to warp or otherwise distort virtual effects256 projected onto object 126 b of real-world venue 120 to conformvirtual effects 256 to curved surface 128 of object 126 b from thepresent vantage point of user 132, i.e., the location of the movingperspective of user 132 at the time of projection of virtual effects256, and to vary that conformal projection in real-time as the vantagepoint of user 132 changes. As a result, virtual-world simulator 100/200can advantageously generate virtual-world simulation 130/230 providinguser 132 with a realistic and highly immersive 3D virtual experiencewithout requiring user 132 to wear an augmented reality AR viewingdevice, such as an AR headset, glasses, or to carry an AR enabledpersonal communication device such as a smartphone or tablet computer.

Referring to FIGS. 1B and 2, in the exemplary implementation shown inthat figure, hardware processor 112/212 of computing platform 110/210may execute virtual effects conforming software code 118/218 a tocontrol projection device 104/204 a to warp or otherwise distort virtualeffects 256 projected onto real-world venue 120 to conform virtualeffects 256 to the 3D geometry of objects within real-world venue 120from the present vantage point of camera 134, and to vary that conformalprojection in real-time as the vantage point of camera 134 changes. As aresult, virtual-world simulator 100/200 can advantageously generatevirtual-world simulation 130/230 enabling transformation of real-worldvenue 120 in the form of a personal residence or photography studio intoa movie set providing a backdrop for a performance by user 132, withoutrequiring user 132 to be present at a movie studio backlot, or even toleave his/her own home.

Referring to FIGS. 1C and 2, in some implementations, virtual-worldsimulator 100C/200 may use tracking system 102 to track the movingperspective of second user 136, while also tracking the movingperspective of user 132, and to generate a virtual-world simulationconforming the virtual effects included in virtual-world simulation130/230 to the geometry of real-world venue 120 from the present vantagepoint of the second moving perspective of second user 136 whileconcurrently providing real-world simulation 130/230 from the presentvantage point of the moving perspective of user 132. As noted above, insome implementations, projection device 104/204 a may be ahigh-frame-rate projector, or a polarized projection device, forexample, configured to project multiple perspectives of virtual-worldsimulation 130/230 onto real-world venue 120 so as to cause thosemultiple perspectives to appear to be produced concurrently in real-timeto user 132 and second user 136.

As further described above by reference to FIG. 1C, in otherimplementations, virtual-world simulator 100C/200 may communicate withuser device 140/240 to provide second user 136 with virtual-worldsimulation 130/230 conformed to the geometry of real-world venue 120from the present vantage point of second user 136 via display 148/248 ofuser device 140/240. In one such implementation, hardware processor112/212 of computing platform 110/210 may execute virtual effectsconforming software code 118/218 a to generate a virtual-worldsimulation for second user 136 by conforming the virtual effectsincluded in virtual-world simulation 130/230 to the geometry ofreal-world venue 120 from the present vantage point of the movingperspective of second user 136. In that implementation, hardwareprocessor 112/212 of computing platform 110/210 may further executevirtual effects conforming software code 118/218 a to transmit thegenerated simulation to user device 140/240 for display to second user136 via display 148/248, while concurrently projecting virtual-worldsimulation 130/230 from the present vantage point of the movingperspective of user 132.

Referring to FIGS. 1D and 2, in some implementations, user device140/240 may be a handheld device configured to project virtual-worldsimulation 130/230 onto real-world venue 120. As noted above, in someimplementations user device 140/240 may be included as a remotecomponent of virtual-world simulator 100D/200, communicatively coupledto computing platform 110/210. In those implementations, user device140/240 may include user projection device 204 b. That is to say, insome implementations the projection device of virtual-world simulator100/200 may be handheld user device 140/240 configured for use by user132 of real-world venue 120. Alternatively, in some implementations,virtual-world simulator 100/200 may be integrated into user device140/240, thereby imbuing user device 140/240 with substantially all ofthe features and functionality of virtual-world simulators 100A-100D/200disclosed in the present application. Thus, in some implementations,action 378 may be performed by virtual effects conforming softwareapplication 218 b, executed by hardware processor 242 of user device140/240, and using user projection device 204 b.

It is noted that, in some implementations, hardware processor 112/212 ofcomputing platform 110/210 may execute virtual effects conformingsoftware code 118/218 a to perform actions 372, 374, and 376 in anyorder, i.e., action 376 may precede action 374, while either or both ofactions 374 and 376 may precede action 372. In other implementations,hardware processor 112/212 may execute virtual effects conformingsoftware code 118/218 a to perform one or more of actions 372, 374, and376 in parallel. i.e., substantially concurrently. It is further notedthat, in some implementations, hardware processor 112/212 may executevirtual effects conforming software code 118/218 a to perform actions372, 374, 376, and 378 in an automated process from which humanparticipation may be omitted. It is also noted that although the presentapplication has focused on the generation of virtual-world simulation130/230 using visual virtual effects 256, in some implementations, audioeffects can also be generated to enhance the verisimilitude ofvirtual-world simulation 130/230.

For example, real-world venue 120 may include an audio systemcommunicatively coupled to computing platform 110/210 of virtual-worldsimulator 100/200 and configured to modulate environmental sounds, aswell as speech by virtual characters, to agree with the relativedistances of the virtual sources of those sounds from the presentvantage point of user 132 and/or second user 136. Such an audio systemmay include multiple audio output devices, such as speakers of differentsizes and power output capabilities for example, distributed throughoutreal-world environment 120. In one implementation, for instance,hardware processor 112/212 of computing platform 110/210 may executevirtual effects conforming software code 118/218 a to control the outputof individual audio system components to enhance the illusion that aparticular visual effect is physically close to, or far from, the useror users engaging with virtual-world simulation 130/230.

Thus, the present application discloses virtual-world simulators andmethods for use by such simulators that overcome the drawbacks anddeficiencies in the conventional art. According to the present novel andinventive concepts, one or more users can advantageously experience avirtual-world that is simulated by conforming virtual effects to the 3Dgeometry of a real-world venue from the vantage point of each user.Moreover, and as a significant improvement over the presentstate-of-the-art, the virtual-world simulation solution disclosed by thepresent application can advantageously be used to provide users withrealistic and highly immersive individualized 3D virtual experienceswithout requiring those users to wear an augmented reality AR viewingdevice.

From the above description it is manifest that various techniques can beused for implementing the concepts described in the present applicationwithout departing from the scope of those concepts. Moreover, while theconcepts have been described with specific reference to certainimplementations, a person of ordinary skill in the art would recognizethat changes can be made in form and detail without departing from thescope of those concepts. As such, the described implementations are tobe considered in all respects as illustrative and not restrictive. Itshould also be understood that the present application is not limited tothe particular implementations described herein, but manyrearrangements, modifications, and substitutions are possible withoutdeparting from the scope of the present disclosure.

What is claimed is:
 1. A virtual-world simulator in a real-world venue,the virtual-world simulator comprising: a computing platform including ahardware processor and a memory storing a software code; a trackingsystem communicatively coupled to the computing platform; and a handhelddevice configured for use by a user in the real-world venue andcommunicatively coupled to the computing platform; the hardwareprocessor configured to execute the software code to: obtain a map of ageometry of the real-world venue; identify one or more virtual effectsfor display in the real-world venue; track, using the tracking system, amoving perspective of one of the user in the real-world venue or acamera in the real-world venue; and control the handheld device tosimulate a virtual-world by conforming the identified one or morevirtual effects to the geometry of the real-world venue from a presentvantage point of the tracked moving perspective.
 2. The virtual-worldsimulator of claim 1, wherein the map is a three-dimensional (3D) map ofthe geometry of the real-world venue.
 3. The virtual-world simulator ofclaim 1, further comprising a mapping device communicatively coupled tothe computing platform, wherein the hardware processor is furtherconfigured to execute the software code to generate, using the mappingdevice, the map of the geometry of the real-world venue.
 4. Thevirtual-world simulator of claim 3, wherein the mapping device comprisesat least one of a mapping camera or a light detection and ranging(LIDAR) device.
 5. The virtual-world simulator of claim 1, wherein themap of the geometry of the real-world venue comprises athree-dimensional (3D) map of the geometry of the real-world venue. 6.The virtual-world simulator of claim 1, wherein the tracked movingperspective is the moving perspective of the user in the real-worldvenue, and wherein the hardware processor is further configured toexecute the software code to: track a second moving perspective of asecond user in the real-world venue; and control the handheld device tofurther simulate the virtual-world by conforming the identified one ormore virtual effects to the geometry of the real-world venue from asecond present vantage point of the second moving perspective of thesecond user while concurrently simulating the virtual-world from thepresent vantage point of the moving perspective of the user.
 7. Thevirtual-world simulator of claim 1, wherein the tracked movingperspective is the moving perspective of the user in the real-worldvenue, and wherein the hardware processor is further configured toexecute the software code to: detect a user device utilized by a seconduser in the real-world venue; track a second moving perspective of thesecond user in the real-world venue; generate a simulation of thevirtual-world by conforming the identified one or more virtual effectsto the geometry of the real-world venue from a second present vantagepoint of the second moving perspective of the second user; and transmitthe generated simulation to the user device for display to the seconduser while concurrently simulating the virtual-world from the presentvantage point of the moving perspective of the user.
 8. Thevirtual-world simulator of claim 1, wherein the tracked movingperspective is the moving perspective of the camera, and wherein thecamera is configured to obtain an image of the user in the simulatedvirtual-world.
 9. The virtual-world simulator of claim 1, wherein thehandheld device is a projection device.
 10. A method for use by avirtual-world simulator in a real-world venue, the virtual-worldsimulator including a computing platform having a hardware processor anda memory storing a software code, the computing platform communicativelycoupled to a tracking system and a handheld device configured for use bya user in the real-world venue, the method comprising: obtaining, by thesoftware code executed by the hardware processor, a map of a geometry ofthe real-world venue; identifying, by the software code executed by thehardware processor, one or more virtual effects for display in thereal-world venue; tracking, by the software code executed by thehardware processor and using the tracking system, a moving perspectiveof one of the user in the real-world venue or a camera in the real-worldvenue; and controlling, by the software code executed by the hardwareprocessor, the handheld device to simulate a virtual-world by conformingthe identified one or more virtual effects to the geometry of thereal-world venue from a present vantage point of the tracked movingperspective.
 11. The method of claim 10, wherein the map is athree-dimensional (3D) map of the geometry of the real-world venue. 12.The method of claim 10, wherein the virtual-world simulator furthercomprises a mapping device communicatively coupled to the computingplatform, the method further comprising: generating, by the softwarecode executed by the hardware processor and using the mapping device,the map of the geometry of the real-world venue.
 13. The method of claim10, wherein the mapping device comprises at least one of a mappingcamera or a light detection and ranging (LIDAR) device.
 14. The methodof claim 10, wherein the map of the geometry of the real-world venuecomprises a three-dimensional (3D) map of the geometry of the real-worldvenue.
 15. The method of claim 10, wherein the tracked movingperspective is the moving perspective of the user in the real-worldvenue, the method further comprising: tracking, by the software codeexecuted by the hardware processor and using the tracking system, asecond moving perspective of a second user in the real-world venue; andcontrolling, by the software code executed by the hardware processor,the handheld device to further simulate the virtual-world by conformingthe identified one or more virtual effects to the geometry of thereal-world venue from a second present vantage point of the secondmoving perspective of the second user while concurrently simulating thevirtual-world from the present vantage point of the moving perspectiveof the user.
 16. The method of claim 10, wherein the tracked movingperspective is a moving perspective of the user in the real-world venue,the method further comprising: detecting, by the software code executedby the hardware processor, a user device utilized by a second user inthe real-world venue; tracking, by the software code executed by thehardware processor and using the tracking system, a second movingperspective of the second user in the real-world venue; generating, bythe software code executed by the hardware processor, a simulation ofthe virtual-world by conforming the identified one or more virtualeffects to the geometry of the real-world venue from a second presentvantage point of the second moving perspective of the second user; andtransmitting, by the software code executed by the hardware processor,the generated simulation to the user device for display to the seconduser while concurrently simulating the virtual-world from the presentvantage point of the moving perspective of the user.
 17. The method ofclaim 10, wherein the tracked moving perspective is the movingperspective of the camera, and wherein the camera is configured toobtain an image of the user in the simulated virtual-world.
 18. Themethod of claim 10, wherein the handheld device is a projection device.19. A virtual-world simulator in a real-world venue, the virtual-worldsimulator comprising: a computing platform including a hardwareprocessor and a memory storing a software code; a tracking systemcommunicatively coupled to the computing platform; and a projectiondevice communicatively coupled to the computing platform; the hardwareprocessor configured to execute the software code to: obtain a map of ageometry of the real-world venue; identify one or more virtual effectsfor display in the real-world venue; track, using the tracking system, afirst moving perspective of a first user in the real-world venue; track,using the tracking system, a second moving perspective of a second userin the real-world venue; and control the projection device toconcurrently simulate a virtual-world by conforming the identified oneor more virtual effects to the geometry of the real-world venue from afirst present vantage point of the tracked first moving perspective anda second present vantage point of the tracked second moving perspective.20. A virtual-world simulator in a real-world venue, the virtual-worldsimulator comprising: a computing platform including a hardwareprocessor and a memory storing a software code; a tracking systemcommunicatively coupled to the computing platform; and a projectiondevice communicatively coupled to the computing platform; the hardwareprocessor configured to execute the software code to: obtain a map of ageometry of the real-world venue; identify one or more virtual effectsfor display in the real-world venue; track, using the tracking system, afirst moving perspective of a first user in the real-world venue;control the projection device to simulate a virtual-world by conformingthe identified one or more virtual effects to the geometry of thereal-world venue from a first present vantage point of the tracked firstmoving perspective; detect a user device utilized by a second user inthe real-world venue; track, using the tracking system, a second movingperspective of a second user in the real-world venue; generate asimulation of the virtual-world by conforming the identified one or morevirtual effects to the geometry of the real-world venue from a secondpresent vantage point of the tracked second moving perspective; andtransmit the generated simulation to the user device for display to thesecond user while concurrently simulating the virtual-world from thefirst vantage point of the tracked first moving perspective.
 21. Avirtual-world simulator in a real-world venue, the virtual-worldsimulator comprising: a computing platform including a hardwareprocessor and a memory storing a software code; a tracking systemcommunicatively coupled to the computing platform; and a projectiondevice communicatively coupled to the computing platform; the hardwareprocessor configured to execute the software code to: obtain a map of ageometry of the real-world venue; identify one or more virtual effectsfor display in the real-world venue; track, using the tracking system, amoving perspective of a camera in the real-world venue; and control theprojection device to simulate a virtual-world by conforming theidentified one or more virtual effects to the geometry of the real-worldvenue from a present vantage point of the tracked moving perspective,wherein the camera is configured to obtain an image of the user in thesimulated virtual-world.
 22. A method for use by a virtual-worldsimulator in a real-world venue, the virtual-world simulator including acomputing platform having a hardware processor and a memory storing asoftware code, the computing platform communicatively coupled to atracking system and a projection device, the method comprising:obtaining, by the software code executed by the hardware processor, amap of a geometry of the real-world venue; identifying, by the softwarecode executed by the hardware processor, one or more virtual effects fordisplay in the real-world venue; tracking, by the software code executedby the hardware processor and using the tracking system, a first movingperspective of a first user in the real-world venue; tracking, by thesoftware code executed by the hardware processor and using the trackingsystem, a second moving perspective of a second user in the real-worldvenue; and controlling, by the software code executed by the hardwareprocessor, the projection device to concurrently simulate avirtual-world by conforming the identified one or more virtual effectsto the geometry of the real-world venue from a first present vantagepoint of the tracked first moving perspective and a second presentvantage point of the tracked second moving perspective.
 23. A method foruse by a virtual-world simulator in a real-world venue, thevirtual-world simulator including a computing platform having a hardwareprocessor and a memory storing a software code, the computing platformcommunicatively coupled to a tracking system and a projection device,the method comprising: obtaining, by the software code executed by thehardware processor, a map of a geometry of the real-world venue;identifying, by the software code executed by the hardware processor,one or more virtual effects for display in the real-world venue;tracking, by the software code executed by the hardware processor andusing the tracking system, a first moving perspective of a first user inthe real-world venue; controlling, by the software code executed by thehardware processor, the projection device to simulate a virtual-world byconforming the identified one or more virtual effects to the geometry ofthe real-world venue from a first present vantage point of the trackedfirst moving perspective; detecting, by the software code executed bythe hardware processor, a user device utilized by a second user in thereal-world venue; tracking, by the software code executed by thehardware processor and using the tracking system, a second movingperspective of a second user in the real-world venue; generating, by thesoftware code executed by the hardware processor, a simulation of thevirtual-world by conforming the identified one or more virtual effectsto the geometry of the real-world venue from a second present vantagepoint of the tracked second moving perspective; and transmitting, by thesoftware code executed by the hardware processor, the generatedsimulation to the user device for display to the second user whileconcurrently simulating the virtual-world from the first vantage pointof the tracked first moving perspective.
 24. A method for use by avirtual-world simulator in a real-world venue, the virtual-worldsimulator including a computing platform having a hardware processor anda memory storing a software code, the computing platform communicativelycoupled to a tracking system and a projection device, the methodcomprising: obtaining, by the software code executed by the hardwareprocessor, a map of a geometry of the real-world venue; identifying, bythe software code executed by the hardware processor, one or morevirtual effects for display in the real-world venue; tracking, by thesoftware code executed by the hardware processor and using the trackingsystem, a moving perspective of a camera in the real-world venue;tracking, by the software code executed by the hardware processor andusing the tracking system, a second moving perspective of a second userin the real-world venue; and controlling, by the software code executedby the hardware processor, the projection device to simulate avirtual-world by conforming the identified one or more virtual effectsto the geometry of the real-world venue from a present vantage point ofthe tracked moving perspective, wherein the camera is configured toobtain an image of the user in the simulated virtual-world.